Capacity of Random Channels with Large Alphabets

We consider discrete memoryless channels with input alphabet size n and output alphabet size m, where m = ⌈γn⌉ for some constant γ > 0. The channel transition matrix consists of entries that, before being normalized, are independent and identically distributed nonnegative random variables V and such that E [ (V logV ) ] < ∞. We prove that in the limit as n → ∞ the capacity of such a channel converges to Ent(V )/E[V ] almost surely and in L, where Ent(V ) := E[V logV ] − E[V ] log E[V ] denotes the entropy of V . We further show that, under slightly different model assumptions, the capacity of these random channels converges to this asymptotic value exponentially in n. Finally, we present an application in the context of Bayesian optimal experiment design.